Variational Autoencoders (VAE)

What is VAE?

= A generative model that learns latent variable distributions.

• extends Autoencoders with probabilistic encoding (Variational Inference)
• enables smooth latent space and generative capabilities

Architecture

• Encoder
• Decoder

How it works

1. Encoder maps input $x$ to parameters of latent distribution $q_{\varphi }(z|x)$ — mean $\mu$ , variance ${\sigma }^2$.
2. Sampling: Latent vector $z$ sampled from this distribution using the reparameterization trick (enables backpropagation through stochastic sampling):

3. Decoder: Reconstructs $x$ from $z$ by modeling $p_{\theta }(x|z)$.

Loss Function

• ELBO (Evidence Lower Bound):

• Terms:
  • Reconstruction loss: How well decoder reconstructs input.
  • KL divergence: Regularizes latent distribution $q_{\varphi }(z|x)$ to be close to prior $p(z)$ (usually standard normal).